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Abstract 

Background: The Theory of Change (ToC) approach has been used to develop and evaluate complex health 
initiatives in a participatory way in high income countries. Little is known about its use to develop mental health 
care plans in low and middle income countries where mental health services remain inadequate. 

Aims: ToC workshops were held as part of formative phase of the Programme for Improving Mental Health Care 
(PRIME) in order 1) to develop a structured logical and evidence-based ToC map as a basis for a mental health care 
plan in each district; (2) to contextualise the plans; and (3) to obtain stakeholder buy-in in Ethiopia, India, Nepal, 
South Africa and Uganda. This study describes the structure and facilitator's experiences of ToC workshops. 

Methods: The facilitators of the ToC workshops were interviewed and the interviews were recorded, transcribed 
and analysed together with process documentation from the workshops using a framework analysis approach. 

Results: Thirteen workshops were held in the five PRIME countries at different levels of the health system. The ToC 
workshops achieved their stated goals with the contributions of different stakeholders. District health planners, 
mental health specialists, and researchers contributed the most to the development of the ToC while service 
providers provided detailed contextual information. Buy-in was achieved from all stakeholders but valued more 
from those in control of resources. 

Conclusions: ToC workshops are a useful approach for developing ToCs as a basis for mental health care plans 
because they facilitate logical, evidence based and contextualised plans, while promoting stakeholder buy in. 
Because of the existing hierarchies within some health systems, strategies such as limiting the types of participants 
and stratifying the workshops can be used to ensure productive workshops. 
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Background 

Mental health services remain inadequate in low and 
middle income countries (LMIC). They are marked by 
low financial investment, insufficient human resources 
and lack of political priority and planning for mental 
health care [1,2]. In order to expand and improve access, 
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it is imperative that mental health is integrated into pri- 
mary health care and other health platforms as well as into 
the services provided by other sectors including education, 
social services, justice and labour [3]. Although evidence 
exists for individual evidence based interventions, less is 
known about how they can be integrated into existing 
health services [4] . Engaging key stakeholders in participa- 
tory planning for mental health services is critical to 
develop such services and resources, get local and national 
stakeholder buy-in, and develop plans that are context- 
ually appropriate [5,6]. 
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Theory of Change (ToC) is a participatory theory driven 
approach to programme design and evaluation whose under- 
lying principle is to improve our understanding of how and 
why a programme works [7]. This is achieved through the 
development of a ToC, or programme theory, which de- 
scribes the causal pathways through which a programme is 
hypothesised to have an effect. The ToC is often developed 
in consultation with key stakeholders in ToC workshops or 
interviews, document review or programme observation [8]. 
Social science, management, sociological or other formal the- 
ories are inserted into the framework to explain why and 
how the causal pathways operate [8,9]. The ToC is often dis- 
played visually as a ToC map [10]. 

The ToC approach was developed from theory driven 
evaluation approaches which include the logical frame- 
works and logic models [11] and has been influenced by 
informed social action approaches [12]. Although often 
used exclusively as an evaluation tool, Connell and 
Kubisch in one of the seminal articles on ToC proposed 
that it be used both in for programme development and 
evaluation [13]. 

Interest in the ToC approach has grown recently in 
the development and NGO sector used by agencies such 
as DFID, Oxfam, and Comic Relief [14] for both pro- 
gram design and evaluation. Despite the abundance of 
guidelines on how to develop a ToC [10,15,16] and its 
widespread use there are few published case reports of 
their application in the academic literature and the 
majority of these describe the use in evaluation of pro- 
grammes and not their design. There are only a few 
examples in the academic literature describing the role 
of ToC in the planning of complex health interventions 
[17,18]. These include the use of ToC in the develop- 
ment and evaluation of mental health systems of care for 
children and adolescents in the US [19,20]. Results from 
these experiences show that ToC can be used effectively 
as a planning tool for implementation as well as provide 
a framework for evaluation [20]. In addition, using ToC 
provides a mechanism for consensus building amongst 
stakeholders and a shared service delivery strategy. 

There is very little detail published on how ToCs have 
been developed. Methods of ToC development reported 
in the literature include review of programme documen- 
tation [21], interviews and focus group discussions with 
key stakeholders [17,22,8], using existing theory or re- 
search [23,24] and ToC workshops [8] but few describe 
their methods in enough detail to replicate the ToC 
development. 

Proponents of ToC advocate for the use of ToC work- 
shops to develop ToCs as they allow participation of vari- 
ous stakeholders who can share knowledge, debate 
specific aspects of the ToC, articulate assumptions, and 
assess the feasibility of the intended interventions in the 
specific context [8,13]. For example, Mason and Barnes 



(2007) used ToC workshops with key stakeholders to 
develop a ToC for the evaluation of the New Children's 
Fund, a multi-agency collaboration to deliver preventive 
services for children. During the workshops, they explored 
the needs of the target group, the short, medium and long 
terms outcomes the programme was working to address, 
the activities through which the outcomes could be 
achieved, the rationale of the activities, and the local and na- 
tional policy context. However, few additional examples of 
ToC workshops have been published in the academic litera- 
ture [25] and, to our knowledge, none use ToC workshops 
to develop an intervention within a health system in LMIC. 

The majority of the guidance on how to conduct ToC 
workshops has been developed by funding and develop- 
ment organisations which outline how ToC workshops 
can be conducted. A common approach starts with stake- 
holders reaching agreement on the intended impact, then 
working backward to determine the intermediate and 
short term outcomes necessary and sufficient to achieve 
the intended impact [10,26]. These outcomes are opera- 
tionalized by identifying indicators for each outcome 
which will determine whether the outcome has been 
achieved. In addition, the evidence base or rationale of 
how one outcome leads to the next is articulated and 
whether an intervention is required to achieve this. Stake- 
holders are encouraged to articulate the assumptions 
underlying the theory as well as to decide a ceiling of 
accountability where the programme is no longer directly 
responsible for the outcomes achieved. The ultimate ToC 
should be plausible, do-able and testable [13] and can be 
represented graphically in a ToC map (Figure 1). 

The programme for improving mental health care (PRIME) 

The Programme for Improving Mental Health Care 
(PRIME) is a multi-country research programme which 
aims to provide evidence for how to integrate mental 
health into primary care by developing, implementing 
and evaluating district level mental health care plans 
(MHCPs) for priority disorders [27]. It is working in 
pilot districts or sub-districts in five LMIC, namely in 
Sodo, Ethiopia; Sehore, India; Chitwan, Nepal; Dr Kenneth 
Kuanda, South Africa; and Kamuli, Uganda (Table 1). 
Mental health service resources vary considerably across 
the district sites. Still, all countries face health systems and 
contextual challenges [28]. Within each district, specific 
packages of mental health care made up of several inter- 
acting components have been developed for implementa- 
tion within three levels of the health system: healthcare 
organisation, health facility and community. The PRIME 
MHCPs target three priority disorders: depression, alcohol 
use disorders and psychosis, with the addition of epilepsy 
in Ethiopia, Nepal and Uganda. One of the key principles 
of PRIME is a partnership between researchers and the 
Ministries of Health in each of the PRIME countries. As 
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Figure 1 Elements of a Theory of Changes (adapted from Andersen 2004). 



part of this partnership, the human resources for the im- 
plementation of PRIME are largely provided by the Minis- 
tries of Health while the researchers provide training, 
technical support and evaluation [29]. As such, the PRIME 
MHCPs meet the criteria for complex interventions as out- 
lined by Craig et al. [29] including multiple groups of 
stakeholders and organisational levels targeted by the inter- 
vention. The intervention achieves multiple outcomes 
through several causal strands. 

The PRIME MHCPs have been developed for each dis- 
trict through formative work including reviews of the lit- 
erature, a situational analysis of mental health care in 
the district [28], semi structured interviews and focus 
group discussions with stakeholders [31]. As part of the 
development of the PRIME MHCPs we used a ToC ap- 
proach which involved the development of a PRIME 
cross-country and district specific ToCs. 

Table 1 PRIME district sites adapted from Lund et al. [29] 



This paper describes how the district specific workshops 
were used in the planning stages of PRIME. Specifically, 
we describe the overall structure and stakeholder compos- 
ition of the workshops and the facilitators' experiences of 
how stakeholders contributed to the three purposes of the 
ToC workshops. The purposes were to 1) develop a logical 
evidenced based ToC map, 2) inform the development of 
a contextualised mental health care plan; and, 3) obtain 
the buy-in of key stakeholders. We further describe how 
these purposes were achieved within a hierarchical health 
system and potential approaches to and limitations of 
mitigating the effects of this hierarchy. 

Methods 

The ToC process in PRIME 

The ToC process began by developing an initial PRIME 
cross-country ToC in a workshop attended by 15 key 



Country 


World 

bank region 1 


World bank 

income 

classification 1 


Gross national 
income 

per capita (USD) 1 


PRIME 

District/ 

sub-district 


Population 2 


Socio-economic 
characteristics 2 


Number of 
Health Facilities 2 


Number of 
MH specialists 2 


Ethiopia 


Sub-Saharan 
Africa 


Low Income 


400 


Sodo 


1 65,000 


Literacy rate = 
22%; 90% rural 


0 hospitals, 1 district 
health bureau, 7 
community health 
centers, 52 health 
posts 


None 


India 


South Asia 


Lower middle 
income 


1410 


Sehore 
(Madhya 
Pradesh 
state) 


1,311,008 


Literacy rate: 
71% 81% rural 


2 hospitals, 8 
community health 
centers, 15 primary 
health clinics 


1 part-time 
psychiatrist, 1 
psychologist 


Nepal 


South Asia 


Low income 


54Q 


Chitwan 


575,058 


Literacy rate = 
70% 73% rural 


152 sub -health 
centers 


2 Psychiatrists 


South 
Africa 


Sub-Saharan 
Africa 


Upper Middle 
Income 


6960 


Kenneth 

Kaunda 

(North 

West 

Province) 


632,790 


Literacy rate: 
88% 14% rural 


2 hospitals, 4 
primary health 
centers, 5 health 
posts 


1 Psychiatrist, 
1 Psychologist 


Uganda 


Sub-Saharan 
Africa 


Low Income 


500 


Kamuli 


740,700 


Literacy rate: 
62% 3% rural 


41 sub-health posts 


1 Psychiatric 
Clinical Officer 



'Countries and Economies [http://data.worldbank.org/country/] [30]. 
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PRIME partners, including two representatives from each 
PRIME country team, in Goa, India in October 2011. The 
workshop aimed to both introduce the PRIME partners to 
the ToC approach and to develop a PRIME cross country 
ToC as a framework for the district level MHCPs. 

The ToC developed during this workshop identified the 
intended impact of the PRIME intervention, namely im- 
proved health, social and economic outcomes in people liv- 
ing with the priority mental disorders in the selected 
districts of PRIME. The workshop participants identified 
the anticipated short, medium and long term goals re- 
quired to achieve the impact across the three levels of the 
health system. The outcomes were identified in the follow- 
ing domains: political buy-in, programme resources, cap- 
acity building, identification and diagnosis of mental 
disorders and service delivery. Participants also identified 
assumptions and gaps in knowledge which informed the 
development of the formative research questions to de- 
velop contextualised MHCPs in each district. The ToC was 
then refined and modified by members of the PRIME con- 
sortium over the following year. An abridged version of the 
ToC showing the outcomes only is illustrated in Figure 2. 
The overarching cross country process of ToC develop- 
ment in PRIME, the resulting ToC map and the influence 



of the ToC on the PRIME cross country evaluation design 
will be described in detail in a subsequent paper. 

Following this, each PRIME country team conducted at 
least two ToC workshops to assist with the development 
of the district specific ToCs. These workshops aimed to: 
1) develop a logical evidenced based ToC, 2) inform the 
development of a contextualised mental health care plan; 
and, 3) to garner the buy-in of key stakeholders. The 
resulting ToCs were used as a 'blueprint' for the PRIME 
MHCPs which were developed further using results of the 
PRIME situational analysis, formative work, costing tool 
and literature reviews. The ToCs were used to validate 
and expand on the PRIME cross country ToC which was 
used as a framework to evaluate the effectiveness of the 
plan following implementation. The structure, number, 
composition and process of the workshops was deter- 
mined by PRIME country teams in line with brief cross 
country guidelines in conjunction with Andersen's guide- 
lines [10]. 

Stakeholders were defined as those involved in the imple- 
mentation of the program, served or affected by the pro- 
gram or using the evaluation results [32]. They were 
purposively sampled and recruited at the discretion of coun- 
try teams who aimed to balance a productive workshop with 
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Figure 2 An abridged version of the outcomes and impact of the PRIME Cross Country Theory of Change. 
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the hierarchical nature of their respective health systems. 
Participants included diverse stakeholders such as district 
health service managers, primary health care service pro- 
viders, district mental health coordinators, members of the 
community leaders, mental health specialists, national level 
policy makers and mental health service users. 

Data collection and tools 

Data about the ToC workshops were collected from a 
number of sources. First we collected process documenta- 
tion from all workshops produced by the PRIME country 
teams in English. This included minutes or workshop 
reports which reported on the key content and structure 
of the workshops and participant lists. 

Secondly, we conducted 5 individual and 5 joint semi- 
structured interviews with 9 facilitators of the ToC work- 
shops (4 principal investigators and 5 project co-ordinators) 
following both the preliminary and final workshops. The 
decision to conduct joint interviews with 2-3 facilitators 
from one country, or individual interviews, was made by 
country principal investigators. The interviews were de- 
signed to elicit the facilitators' experiences of the workshops 
and stimulate discussion on the practical aspects of how 
the workshops were conducted namely stakeholder com- 
position, workshop structure, group dynamics, the useful- 
ness of the process and to generate lessons for future use in 
intervention development. 

All interviews were telephonically or face-to-face con- 
ducted by the first author (EB) in English and tran- 
scribed by a professional transcriber familiar with health 
research. The transcripts were checked for accuracy by 
EB. Additional information was gained through email 
correspondence, informal discussions and presentations 
at PRIME consortium meetings as well as direct experi- 
ence of the workshops by co-authors. 

Data analysis 

A framework analysis approach was used to analyse the 
process documentation and interview transcripts [33]. 
This method was developed for applied policy research 
and contains five key stages: familiarisation, identifying a 
thematic framework, indexing, charting, mapping, and 
interpretation. Qualitative data software, NVivo9, was 
used to assist with the analysis [34]. 

Following familiarisation with the process documenta- 
tion and interview transcripts, a coding framework was 
developed by EB based on the semi structured interview 
guide. The main themes included workshop structure, 
participants, dynamics and emerging themes. A frame- 
work matrix was generated using the themes: workshop 
structure; participants and dynamics; and their 25 sub- 
themes which mapped onto the X axis of a spreadsheet. 
The 13 ToC workshops were mapped onto the Y axis. 
The coded data in each cell were summarised to reflect 



the content. The data within each column or sub-theme 
was compared across ToC workshops and interpreted. 
As the data were compared additional salient themes 
emerged. Following the main analysis by EB, the results 
were summarised and validated by co-authors who were 
part of the country workshops. 

The study was approved by the Human Ethics Re- 
search Committee of the Faculty of Health Sciences at 
the University of Cape Town (REC Ref:247/2013). Ethical 
approval for the PRIME research programme has also 
been obtained by local ethics committees in study coun- 
tries. All those who participated in the semi-structured in- 
terviews gave informed consent to participate. 

Findings 

Between two and four ToC workshops were held in each 
country at different levels of the health system to de- 
velop ToCs for the PRIME district site, with a total of 13 
workshops across the 5 countries. Table 2 outlines how 
each workshop was structured and Table 3 describes the 
stakeholders who participated in the workshops. More 
detail about the structure, stakeholders and how the 
workshops were conducted to achieve their multiple 
goals are described in subsequent sections. 

ToC Workshop structure 

The ToC workshops were structured to include a wel- 
come, a brief introduction to PRIME, a discussion on 
mental health (in some cases this was discussed in a 
pre-workshop meeting) and an introduction to the ToC 
approach. In India, Uganda and Ethiopia, the workshops 
were introduced by state or national Ministry of Health 
representatives. 

Most countries started from the impact and worked 
backwards to map long, medium and short term out- 
comes required to achieve the impact. When developing 
the initial TOC map, workshop participants in all coun- 
tries except South Africa developed a generic ToC map 
for mental illness rather than developing separate ToCs 
for each priority disorder (depression, alcohol use, 
psychosis and epilepsy). This was because the workshop 
facilitators hypothesised that the causal pathway through 
which the integration of mental health into primary care 
leads to improved outcomes were essentially the same 
for different disorders. The generic map was then com- 
pared and modified for the different disorders with very 
few changes needed for the specific disorders. 

Stakeholders 

The stakeholders were selected using a variety of criteria 
including: 1) involvement in planning or implementing the 
PRIME MHCPs at various levels including providing spe- 
cialist care, co-ordinating services, providing primary care 
services, managing facilities or developing and evaluating 
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Table 2 Summary of PRIME ToC workshops 



Country 


Level 


Location 


Length 


Structure 


Ethiopia 


Workshop 
T.(ETI) 


Community and district level 
representatives 


Sodo* 


Vi day 


a Introduction to PRIME 

b Explanation of the ToC process 

c Agreement on impact 

d Worked forwards to development of the ToC discussing current services, 
needs and potential outcomes of the ToC to reach the desired impact. 


Workshop 
2. (ET2) 


National level planners 


Addis 
Ababa 


1 day 


a Introduction of Ethiopian mental health strategy by national ministry of 
health representative 

c Introduction to ToC and the ToC process 

c Review and refinement of the ToC developed in ET1. 


India 


Workshop 
1. (INI) 


District and health Facility 


Sehore* 


1 day 


a Introduction to mental health and PRIME 

b Introduction to ToC 

c Mental health presentation 

d Group work where each group developed the outcomes pathway for the 
ToC 

e Feedback from group work 


Workshop 
2. (IN2) 


District and health Facility 


Sehore* 


1 day 


a Summary of IN1 

b Group work: details of interventions and assumptions at community, 
health facility and health organisation level in the existing ToC map. 

c Presentation and discussion of the integrated mental health care plan 
developed from the ToC. 


Nepal 


Workshop 
1. (NE1) 


Health Facility and District 


Chitwan* 


1 day 


a Introduction to PRIME 
b Introduction to ToC 

c Agreement on long-term impact and worked the group agreed on the 
long term impact then worked backwards to determine the outcomes, inter- 
ventions and assumptions needed to achieve this. 


Workshop 
2. (NE2) 


National level planners 


Kathmandu 


Vi day 


a Introduction to PRIME 
b Introduction to ToC 

c ToC from NE1 was presented, reviewed and refined by the group. 


Workshop 


Health Facility and District 


Chitwan* 


Vi day 


a Review of the ToC developed in NE1 and NE2 


3. (NE3) 








b Discussion of potential adaptation for specific disorder and indicators to 
measure outcomes. 


Workshop 


National level planners 


Kathmandu 


Vi day 


c Review of the ToC refined in NE3 


4. (NE4) 








d Discussion of potential adaptation for specific disorders and indicators to 
measure outcomes. 


South 
Africa 










Workshop 
1. (SA1) 


Health facility, district, provincial 
and national level representatives 


Dr Kenneth 
Kuanda* 


2 days 


a Introduction to PRIME 
b Introduction to ToC 

c Used part of the PRIME cross country ToC and worked forward adding 
detail to each outcome for all four disorders. 


Workshop 
2. (SA2) 


Community 


Dr Kenneth 
Kuanda* 


1 day 


a Introduction to PRIME 
b Introduction to ToC 

c Used part of the PRIME cross country ToC and worked forward adding 
detail to each outcome for all four disorders. 


Workshop 
3. (SA3) 


Health facility, district, provincial 
and national level representatives 


Dr Kenneth 
Kuanda* 


1 day 


a SA1 workshop was reviewed briefly. 
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Table 2 Summary of PRIME ToC workshops (Continued) 



b Disorder specific integrated mental health care plan based on SA1 was 
presented and discussed in detail. 

c PRIME evaluation plan and next steps were discussed. 



Uganda 



Workshop District and health facility level Kamuli* 1 day a PRIME, mhGAP, challenges for mental health care and the ToC were 
1.(UG1) introduced. 

b The impact was agreed on and the group worked backwards to develop 
the ToC. 

Workshop District and health facility level Kamuli* 1 day a The group was oriented to the ToC process, PRIME and planned work. 



2. (UG2) 



b The ToC map from UG1 was reviewed and refined. 



Locations marked with * indicate the PRIME district in the respective countries. 



the MHCPs; 2) membership of PRIME Community Advis- 
ory Boards who provide an oversight function to PRIME at 
district level; 3) specialist knowledge of mental health; 4) 
representatives of service users or other sectors in the 
wider community; or 5) decision making power or control 
over resources. 

The size of the workshops varied considerably across 
countries with a median of 15 (Interquartile range 13 - 
22) stakeholders attending each workshop (Table 3). 
Most countries held preliminary and final workshops 
with the same group of people, comprising stakeholders 
at different levels of the health system (Table 2) with the 
exception of Ethiopia where their first workshop in- 
cluded district representatives and the second national 
level representatives and mental health specialists. Some 
countries, such as Uganda, India and South Africa relied 
on key individuals to assist with the identification of par- 
ticipants. These were often the District Medical Officers 
who assisted by inviting the participants to the workshop 
and thus providing the workshop with local legitimacy. 

The stakeholders attending the workshops varied by 
country and by workshop (see Table 3). However, five 
key groups of stakeholders attended all workshops: pol- 
icy makers; district level health planners and manage- 
ment; mental health specialists; researchers; and service 
providers. These groups were not mutually exclusive and 
many stakeholders belonged to more than one category. 
Some countries also had representation from community 
or non-governmental organisations (NGOs) but there 
was very little mental health service user representation 
with only 3/13 workshops including mental health ser- 
vice users. 

A major potential barrier to stakeholder participation 
in the workshops was the hierarchical nature of local 
health service organisation which would have inhibited 
participation by lower level staff. Consequently, the 
PRIME country teams who facilitated the workshops 
attempted to mitigate the effect of the hierarchical struc- 
tures by stratifying the workshops and holding separate 
workshops at different levels of the health system or by 



limiting the levels of the participants in the workshops. 
For example, the Nepalese and Ethiopian facilitators 
stratified their workshops into district and national level 
workshops while South Africa held a separate workshop 
at the community level. Indian facilitators specifically 
chose to limit their workshop to district level and senior 
facility level in order to prevent power differentials and 
to optimise planning. According to one of the facilita- 
tors, this resulted in everyone "participating because 
there was no hierarchy, they were all district level, all 
sub district level officers". This is in direct contrast to the 
start of the workshops where senior district and state 
level stakeholders were asked to open the workshop: 
"when the four of them were in the room, I think no-one 
was speaking anything they cannot, I mean even if they 
wish to they cannot speak... once all these four people 
were out and then all of a sudden everyone was 
speaking". 

Achieving the goals of the TOC workshops 

The ToC workshops had 3 main goals: development of a 
ToC map to reflect the structure of the proposed district 
MHCP; contextualisation of the MHCP; and ensuring 
the engagement and buy-in of key stakeholders to the 
MHCP. We describe how these goals were achieved 
below. 

1) The development of a structured, logical and evidence- 
based ToC map 

The ToC workshops helped to develop a structured and 
logical ToC which was described by facilitators as a "vis- 
ual map" which, "like a map of the city", they could refer 
to when thinking about their MHCP. 

In four of the five countries, the ToCs were developed 
during the workshop with stakeholders agreeing on the 
intended impact of the PRIME MHCPs and then working 
backwards to determine the outcomes needed to achieve 
this impact. In South Africa, instead of developing a ToC 
from scratch, facilitators used the basic building blocks of 
the cross country ToC to initiate the discussion. They 
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Table 3 Number and category of workshop participants in the ToC workshops 



Country 



Ethiopia India 



Nepal 



South Africa 



Uganda 



Abbreviation 
Number of participants 
Category of stakeholders 

1. Policymakers 

National* Health representatives 
State/province Health representatives 

2. District level planners and management 

District Health representatives 
Health planners/managers 
District Medical officers 
MH coordinators 
Other health coordinators 
Other district administrative or finance staff 
Other district representatives (Justice, Education) 

3. Specialists 

Psychiatrists 
Psychologists 
Psychiatric clinical officers 
Psychiatric nurses 
Other Medical Specialists 

4. Service providers 

Community health center, primary health center and 
sub health posts 

Clinic managers 

Medical officers 

Clinical officers 

Health Assistants 

Nurses 

Lay Health workers (clinic based) 

Lay Health workers (community based) 

Other clinic staff 

5. Researchers 

PRIME 

6. Community and civil society 

NGO/development organisations 
Community 

Community leaders 

Media 

Faith leaders 
Traditional healers 

7. Mental Health Service users 



ET1 

1/ 



ET2 

13 



INI 

20 



IN2 

1/ 



NE1 

14 



NE2 

10 



NE3 NE4 SA1 



SA2 

26 



SA3 

37 



UG1 

22 



UG2 

22 



"Members of the PRIME Consortium. 
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asked stakeholders to comment on the validity of this 
ToC and then used this to elicit more detail from the 
stakeholders. 

Facilitators reported the process of working with the 
group to map out the long, medium and short term out- 
comes which helped to reach consensus as they had to 
work with stakeholders to "refine and redefine and,... 
eventually agree". It also encouraged facilitators and stake- 
holders to focus on outcomes rather than interventions. 
This was a change from usual practice, as one facilitator 
from India observed: "most of us in the field of develop- 
ment... or public health [are] very focused on ... interven- 
tions and activities". Assumptions underlying the ToC 
maps were discussed in all countries, however the primary 
focus was, as a Ugandan facilitator noted, more "on the 
process and the outcome more than... the assumptions". 

The rationale, or evidence base, underlying the path- 
ways on the ToC map, and the indicators used to deter- 
mine success, were seen as the domain of the 
researchers. When the rationale was discussed, it was 
only in relation to whether interventions would be re- 
quired. Most facilitators thought that sourcing the evi- 
dence base underlying the ToC was the role of the 
researchers, as a Ugandan facilitator noted, "because, 
that's about literature, evidence, and that is for us 
really". Similarly, some countries discussed indicators in 
their final workshop, however, this was seen as some- 
thing which was more important for the researchers: "it's 
very important for [the researchers] to know that every- 
thing has been covered and to be able to evaluate the 
plan using indicators" although it may not be "necessary 
for everyone who was attending that ToC workshop". 
Therefore many country teams added the indicators 
once the workshops were completed. 

The planned interventions were discussed in all coun- 
try workshops. Some countries focused on this in detail 
and added additional elements to the ToC workshops. In 
India, South Africa and Ethiopia, facilitators probed 
more into the resources required to implement the 
MHCP and the roles and responsibilities of service pro- 
viders in the intervention. As one South African facilita- 
tor explained, 

"using the TOC process is really important in 

order to be able to enact the TOC plan, we need these 
resources in place and this is what each of these 
resources are going to be doing and this is how we're 
going to capacitate them in order to fulfil their roles 
and responsibilities" . 

This was particularly important in many PRIME coun- 
tries as no new human resources were being made avail- 
able to implement the plan, apart from those already 
available in the district. 



In South Africa and India, facilitators provided add- 
itional detail during the final workshops by presenting 
an integrated mental health care plan based on the pre- 
liminary ToC workshops. A facilitator from India noted 
that "participants were very happy to see that what they 
had done, in the first workshop... came out in a very 
refined manner and very systematic manner". A South 
African facilitator reported a similar experience, 

"we came on board with stuff that people had already 
discussed and agreed on and there was some 
clarifications that were made, a few additions that 
were made and in essence when we came to the 
workshop, there was already agreement that had been 
reached with the previous workshop so this is just 
consolidating what we had on paper and I felt that 
everybody was moving in the same direction". 

2) Contextualising the mental health care plans 

The second purpose of the workshops was to ensure 
that the ToC and the MHCP were contextually relevant. 
During the workshops, the researchers gained contextual 
knowledge in several domains including the functioning 
of district health services, planning for mental health 
programmes, physical resources, medication provision, 
human resources, stigma, cultural understanding of 
mental illness and the existing community structures. 

Stakeholders identified challenges, needs and potential 
solutions. For example, the provision of psychotropic 
medication was identified as a challenge in the Nepal 
district level workshop. Although a steady supply of psy- 
chotropic medication is necessary for effective treatment 
of severe depression and psychosis, no antipsychotic 
medications are on the free drug list and supply of medi- 
cations is irregular with frequent stock outs. Policy 
makers at the national level workshop were able to pro- 
vide potential solutions to this problem including agree- 
ing to provide psychotropic medications in the area of 
implementation of the PRIME MHCP and suggesting 
that additional stock is ordered as a buffer and procure- 
ment processes for emergency supplies are put in place. 
Similarly, during the final workshop in South Africa, 
stakeholders identified the need for psychologically trained 
supervisors for lay counsellors providing psychosocial in- 
terventions. They suggested that intern psychologists 
could be made available in the short term with a long term 
view to lobby the Department of Health to create new 
posts for graduates with a Bachelor of Psychology in 
Counselling degree (BPsych Counselling). 

3) Obtaining stakeholder buy-in 

The third purpose of the workshops was to obtain buy-in 
from the stakeholders on the ToC and MHCPs. As one 
facilitator remarked, one can "have a beautiful TOC map 
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but if [one doesn't] have the buy-in and ... the various hu- 
man resources available to make it work it's not... going to 
work". Buy-in was achieved through the process of devel- 
oping the map in consultation with stakeholders. This re- 
sulted in a sense of ownership where "the people who were 
there in both the workshops feel that it's their product". This 
buy-in, where "the district has owned the theory of change", 
was felt to be an important contribution of the workshops. 
A South African facilitator noted, "the most important 
thing to derive from that second workshop was to get con- 
sensus and agreement from particularly the decision makers 
about who would do what... we need people to buy into... 
these new roles and responsibilities because it does mean 
quite a shift." However, facilitators noted that this buy- in 
may not be achieved if used to cover a larger population. 
One facilitator from India cautioned that "a lot of effort 
need to be taken before theory of change can be used as a 
routine tool for scaling up of programme planning". 

There was general recognition that buy-in was neces- 
sary from those who were in "positions of decision mak- 
ing and affect availability of services and resources". But, 
as a Ugandan facilitator noted, this buy-in may be that 
of individuals who "may not have the power or the polit- 
ical will to change what you need changed." One facilita- 
tor cautioned that it was important to have "both a top 
down and bottom up approach" where "political credibil- 
ity" was provided by national and state representatives 
and service providers "being part of that process was 
really important" so that they could "see that it will ac- 
tually be part of their work that they do." However, it 
was often not possible to have all stakeholders present. 

The contributions of stakeholders to the goals of 
workshops 

The five key groups of stakeholders who contributed in 
each country to at least one of the workshops (mental 
health specialists, researchers, policy makers, district 
level health planners and management, and service pro- 
viders), contributed in different ways to achieve the 
three goals of the ToC workshops. 

Mental health specialists and researchers provided de- 
tails on technical issues such as functioning of existing 
mental health care provision in the district, the need for 
prioritisation of additional disorders (such as epilepsy in 
Nepal) and the provision of feasible and evidence based 
interventions. The researchers, who were facilitating the 
workshops and were often also mental health specialists, 
provided the technical knowledge of the ToC process to 
develop a logical evidence based map. They also pro- 
vided much of the evidence base underlying the inter- 
ventions and the indicators for the ToC which were 
often developed after the workshop. 

The policy makers and national level planners made 
higher level contributions on the structure of the MHCP 



and possible solutions to issues such as medication pro- 
curement in Nepal and supervision structures in Ethiopia. 
They did not provide much additional information on the 
structure of the ToC map or the details of the MHCPs 
when separate policy maker workshops were held in 
Nepal and Ethiopia as these had been provided in the dis- 
trict level workshops: "there wasn't much when it comes to 
high level ...there weren't many changes from the first one, 
it was like... reaching a point of saturation." Buy-in from 
policy makers both prior to and during the workshops 
was essential as they control resources, for example "they 
are responsible for planning all ... health care" and "allo- 
cate budget and programme in government health system." 
The support of policy makers, who often introduced the 
workshops when countries only held them at one level, was 
seen as a way of legitimizing the PRIME project and the 
ToC workshop. They provided "political credibility" and, in 
some cases such as India and Uganda, were the reasons the 
facilitators felt that the workshops were so successful. 

District level health planners and management were the 
main contributors to developing the overall structure of the 
ToC map in most of the workshops as well as providing 
contextual information on what they felt could or could 
not work. This included identifying current challenges, 
needs and potential solutions. For example, in Nepal they 
identified constraints such as incentive structures for volun- 
teers and medication shortages, whereas stakeholders in 
Uganda identified the low priority of mental health and the 
stigma towards service providers who work with people 
with mental illness. In the South African and Ethiopian 
workshops they identified additional community workers 
who could potentially be utilised for PRIME. 

Service providers assisted with providing detailed in- 
formation about the context and the functioning of 
existing systems including current workloads of 
personnel. As such they could comment on their ability 
to take on additional tasks envisaged by the MHCP. As 
described above, their input and buy-in was seen as es- 
sential as they would predominantly be providing the 
services outlined in the MHCP. 

The contribution of stakeholders to the workshop was 
moderated by the presence of other stakeholders who were 
considered higher up in the health system hierarchy. This 
depended on the strength of the hierarchy which was con- 
sidered particularly strong in Nepal and India. This led a fa- 
cilitator from India to remark that the "idea that ToC could 
involve everyone from health policy makers to planners to 
providers to community health workers in one session... 
needs to be kind of retested because it cannot be participa- 
tive in government structures which run on hierarchy". 

Discussion 

In this study, we describe how district specific ToC work- 
shops were used to plan for the integration of mental 
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health services into primary health care in five low re- 
source settings. Comparing workshops across the five 
countries working in PRIME has allowed us to distil some 
key lessons on the use of ToC workshops for complex 
mental health intervention development and reflect on 
these in relation to existing literature. 

We found that ToC workshops provided a useful ap- 
proach to developing a logical structure for mental 
health plans and provided contextual details for imple- 
menting these in district sites and obtaining stakeholder 
buy-in. The participatory nature of the ToC workshops 
allowed stakeholders to work together in a structured 
forum to map out the ToC for the district and creating a 
forum for knowledge exchange and dialogue about needs 
and potential solutions. 

In this process, the power relationships between the 
stakeholders was critical, as confirmed by previous re- 
search that shows that all actors within health services can 
exert different power over implementation of health policy 
[35] and that service providers may choose to exercise this 
power to both promote or hinder implementation [36]. 
Therefore the active participation and buy-in of all stake- 
holders is likely to increase the chances of successful im- 
plementation [6]. This is particularly important in the 
context of mental health services in LMIC where stigma is 
high [37], human resources for health are limited [38], 
funding is minimal [2] and political priority is low [1]. 

From the outset it was clear in some countries that 
hierarchies within the health system would make it diffi- 
cult for district level planners and service providers to 
participate despite using facilitation techniques. As their 
input was seen as essential to the process of the develop- 
ment of the ToC and contextualising the MHCP, country 
teams used various strategies to mitigate these hierarch- 
ies. These included: 1) stratification of stakeholders by 
having separate workshops for service providers and for 
policy makers; 2) limiting participants to a homogenous 
group of stakeholders; or 3) seeking high level buy-in in 
other forums, for example, interviews. Although these 
strategies seem to have increased participation, the ToC 
is no longer 'owned' by all potential stakeholders as is 
recommended by the Aspen Institute [39]. This is simi- 
lar to the finding by Sullivan H and Stewart M [40] that 
it may not always be feasible to achieve total ownership 
of a ToC where all stakeholders are involved in the plan- 
ning and development of a ToC. They propose that 
ToCs may be owned by different groups of stakeholders 
including the evaluators, by a dominant stakeholder, the 
community or an elite group of implementers. 

Other aspects of the ToC process also reduced the 
ownership by all potential stakeholders such as the lack 
of beneficiaries of the program as well as the finalisation 
of the ToC by the PRIME researchers. Mental health ser- 
vice users were present in only 3 of the 13 workshops. 



Although most facilitators would have liked to included 
mental health service users as beneficiaries of the 
programme who can provide an alternative perspective 
on mental illness and care [41,42] they were not in- 
cluded in most workshops because there are currently 
limited or non-existent mental health services in PRIME 
district sites and no active advocacy groups for mental 
health service users [30]. PRIME researchers were in- 
volved in finalising some aspects of the ToC after the 
workshop such as the rationale and indicators without 
involving the whole group of stakeholders included in 
the ToC. Therefore, despite including quite a broad 
range of stakeholders (see Table 3), the ownership of the 
ToCs in PRIME countries most closely resembles what 
Sullivan and Stewart (2006) refer to as elite ownership of 
the ToC: ownership by a small group of leaders includ- 
ing community leaders who are involved in setting up 
and implementing the program. Sullivan and Stewart 
(2006) propose that the ToC process might still be ef- 
fective as these stakeholders often have access to signifi- 
cant resources needed to support and implement wider 
systematic change. 

It is difficult to ascertain post-hoc whether it would 
have been possible in these settings to run a workshop 
with all identified stakeholders or how the stakeholder 
composition of the workshops has affected the resultant 
quality and validity of the ToC. Certainly, the inclusion 
of multiple levels of stakeholders in the ToC workshops 
enabled a combination of top down and bottom up ap- 
proaches to planning by either acting as a structured 
forum for discussion where all stakeholders participated 
in the same workshop or as a conduit between policy 
makers and service providers where workshops were 
stratified. The ToC workshops enabled district level 
stakeholders to directly influence the planning process 
which was then vetted by the policy makers who agreed 
to implement the plan. Undoubtedly the initial success 
PRIME has had in facilitating the bottom up planning 
process was directly influenced by the participation of 
Ministry of Health partners in the consortium forma- 
lised through Memoranda of Understanding and on- 
going policy engagement [27]. However, it is yet to be 
established whether this participatory process has re- 
sulted in real ownership of the MHCPs on the ground 
by service providers and a real increase in resources 
from senior policy makers. 

A key limitation of the workshops was the lack of ex- 
plicit focus on the assumptions underlying the ToC in 
the workshops. Assumptions are seen as one of the core 
elements of ToC which allow stakeholders to ensure that 
they understand each other's perspectives [12]. These 
were not covered in detail, as the facilitators wanted to 
focus more on the outcomes and interventions and felt 
the assumptions may have been too complex for some 
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of the stakeholders. However, the rich discussions 
reflected in the content of the workshops indicated that 
assumptions did emerge during the discussions between 
stakeholders. 

There were several shortcomings of this study. First, 
our sample size was small and we focused only on the 
experiences of workshop facilitators, both in the inter- 
views and in the process documentation produced by re- 
search teams. Some of these facilitators were also 
included as authors on this paper and the remaining au- 
thors were involved in supporting the facilitators in con- 
ducting and refining their ToCs. This may result in a 
biased view of the ToC workshops and an overesti- 
mation of their usefulness. In future it would be useful 
to gauge the extent of buy-in from other stakeholders 
and examine this with the ToC process over the course 
of the project to determine the impact of the long term 
influence of the ToC process. Secondly, we did not ex- 
plicitly examine the power relationships within the 
workshops. For example, the researchers may have been 
seen as powerful "experts" within the field which may 
have prevented frank discussion amongst stakeholders 
and a social desirability bias in the workshop partici- 
pants. Thirdly, we did not explore the roles and contri- 
butions of mental health service users to the ToC 
process which are likely to have been different from 
other stakeholders. Finally, this paper focused on a small 
aspect of the ToC process within PRIME, namely the 
district specific ToC workshops. A more detailed de- 
scription and analysis of the overall ToC process within 
PRIME, including the role of the ToC in the develop- 
ment of the PRIME MHCPs and the evaluation design is 
necessary and planned in a subsequent paper. Despite 
these limitations, we were able to draw a rich compari- 
son of experiences across countries who had quite simi- 
lar experiences across settings and draw on some key 
lessons for conducting ToC workshops within the health 
system in LMIC: 

1. The goals of the workshops should be clearly stated 
prior to the workshop. This should include a 
statement about the level of detail required in the 
workshops and resulting ToC, as well the ideal 
ownership of the ToC and potential limitations 
thereof. 

2. The number, length, structure, components of the 
ToC and stakeholder composition should be flexible 
and adapted to ensure the ToC workshops can meet 
the stated goals within the context. 

3. Facilitators need to be aware of the health system 
hierarchies and composition of workshops should be 
balanced to manage these using facilitation or 
stratification to ensure the ToC can meet the 
stated goals. 



4. Additional strategies such as individual interviews or 
reviews of the resulting ToCs may be necessary to 
involve stakeholders not included in the workshops 
to ensure broader ownership of the ToC. 

5. The support of policy makers is important 
throughout the process to add legitimacy to the 
workshops and increase the likelihood of 
implementation of the resulting MHCP. 

Conclusions 

This study has shown how ToC workshops can be con- 
ducted to develop ToCs as a basis for contextualised 
district level MHCPs and to facilitate stakeholder buy-in. 
The ToC workshops in PRIME demonstrated that differ- 
ent stakeholders contribute different perspectives to the 
planning process and although a wide range of stake- 
holders should be included, hierarchical health systems 
may limit the participation of all stakeholders in the work- 
shops. Various strategies may be required to mitigate these 
effects to achieve the stated goals of the workshops. How- 
ever, these may limit the ownership of the ToC. 
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